Skip to content

feat(workflow-code): workflows and workspace as code - #10129

Open
waleedlatif1 wants to merge 961 commits into
stagingfrom
feat/workspace-code
Open

waleedlatif1 wants to merge 961 commits into
stagingfrom
feat/workspace-code

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Supersedes #10128: this PR now carries workflows as code and workspace as code together, both already reviewed.

Summary

Workflows as code

  • A workflow can be read as typed TypeScript and applied back: sim workflows source get | create | apply with --dry-run, a generated .sim/AGENTS.md and cookbook, and stable exit codes.
  • @sim/workflow-code: compiler (TypeScript → canonical model), printer, materializer (canonical model → stored graph, keeping block ids through a lock file), diagnostics with file:line frames and hints, size and depth limits.
  • @sim/workflow-sdk: the typed SDK generated from the block catalog (agent, condition, table.queryRows, trigger.schedule, …) with workspace resources by stable key (table('tickets'), credential('slack'), skill('tone'), customTool('lookup')).
  • v2 API: GET/POST /api/v2/workflows/{id}/source, create, the source catalog and resources manifest. Every route goes through an authorized application use case; writes are rate limited, refuse workspace API keys, and answer 409 when the draft changed since the read.
  • Code tab: a read-only TypeScript view of the open workflow that refreshes as the canvas or a collaborator changes it.

Workspace as code

  • A directory describes a workspace — tables/, knowledge/<key>/ (settings + docs/), files/, tools/, skills/, workflows/<key>/ — and sim workspaces source get | apply --dir reads it back or applies it in one run, including a workflow and the table, knowledge base, tool or skill it uses.
  • @sim/workspace-code: one codec per kind, a statically parsed declaration reader (never executed), the base file, limits and diagnostics in the same shape. SDK subpath @sim/workflow-sdk/workspace (defineTable, column.*, defineKnowledgeBase, defineTool).
  • Server: GET/POST /api/v2/workspaces/{id}/source and a knowledge document download route; every write goes through the existing domain use case and its authorization.
  • Safety: a three-way rule (local / base / server) writes only what the directory changed and answers 409 when the server changed it too; nothing deleted on the server comes back; deletes need --allow-delete, show what they lose, re-check the resource right before writing, and are refused while a stored workflow still uses it; binary files and documents travel in bounded, resumable rounds with hashes checked before anything is written; a partial apply converges on re-run.

Changes outside the feature flag

  • References inside a JSON string of a JSON code field (e.g. {"summary": "<agent.content>"} in a Response block or table row data) are now JSON-escaped; any value with a newline or quote used to break the JSON.
  • A buffered knowledge document upload now hands its processing to the dispatcher right after the registering commit (the same backend choice the upload-session path makes), in the background; the outbox row stays the durable fallback.
  • The row-locked recheck before a source write compares edges as a set (the edges read has no ORDER BY), removing a spurious 409.

Everything else is behind the global workflow-code flag (off by default). Migrations 0460/0461 add two tables; additive only.

Type of Change

  • New feature
  • Bug fix

Testing

  • Round-trip corpus over every block type at 100% on all properties; property and hostile-input tests for compile, canonicalize, materialize, print and every workspace codec.
  • Real-Postgres integration suites for workflow source (concurrency and staleness, locks, permissions, credential and env visibility, governance) and for every workspace kind plus a cross-kind suite (stale 409s, delete gates, in-use refusals, partial-failure convergence, blob rounds, permissions, limits).
  • HTTP e2e for workflow source read/apply, the workspace source CLI (get → edit → apply → get), and a two-page Code tab UI e2e.
  • Agent dogfooding: agents built the same backends through code vs. today's CLI — the workflow path with the cookbook used ~42% fewer tokens and finished ~5x faster; the workspace path used ~2.75x fewer calls in half the time, caught a column rename before it shipped, and lost no data on a destructive-instruction trap.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Nov 19, 2026 1:28am UTC

Request Review

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 152 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

Comment thread packages/workspace-code/src/layout.ts Resolved
Comment thread apps/sim/package.json Resolved
@greptile-apps

greptile-apps Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 3/5

[High impact] When enabled, source apply can change workflow drafts and many kinds of workspace resources.

Fix both retry paths before merging so a partial apply can finish without manual repair.

Findings

  1. P1 Partial deployments block retries ▶
  2. P1 Interrupted replacements cannot resume ▶

Summary

Adds typed workflow and workspace source directories, CLI/API read-and-apply flows, and a read-only Code tab behind workflow-code.

  • Workflow drafts can be read and applied as typed TypeScript source.
  • Workspace resources can be described in files and applied together.
  • The workflow panel can show the draft as read-only TypeScript.
  • Embedded JSON values and buffered document uploads follow updated shared paths.

Two recovery gaps remain: deployment updates can reject their own partial work as stale, and interrupted MCP replacements cannot resume.

The earlier declaration-test concern was withdrawn after waleedlatif1 explained that these tests check generated SDK files outside the normal type-check.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Source directory] --> B[Parse and plan]
  B --> C{Dry run?}
  C -->|Yes| D[Show changes and errors]
  C -->|No| E[Take workspace lease]
  E --> F[Recheck and write resources]
  F --> G[Return completed changes and base]
  G --> H[Save source files locally]
Loading

Reviews (18) · Last reviewed commit: "Merge remote-tracking branch 'origin/sta..." · Reviewed by Greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found and verified against the latest diff

Confidence score: 3/5

  • table.ts accepts declarations with more than 100 options, so they parse successfully but fail when applied. Enforce MAX_SELECT_OPTIONS during parsing.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/workspace-code/src/kinds/table.ts">

<violation number="1" location="packages/workspace-code/src/kinds/table.ts:168">
P2: The codec accepts more than the server's 100-option limit, so declarations with 101+ options parse successfully and only fail during apply. Enforce `MAX_SELECT_OPTIONS` while parsing the declaration.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 171 files

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 173 files

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 173 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

workspaces source check and apply on a directory holding no kind folder (a
mistyped --dir) passed with nothing checked or applied, and on a workflow folder
did the same; both now refuse and name the right command. workflows source check
on a workspace root names the workspace check instead of asking for a get.
Column names that differ only in case (E_TABLE_DUPLICATE_COLUMN), the column
cap at and past maxColumns (E_TABLE_TOO_MANY_COLUMNS), file paths that differ
only in Unicode normalization, an empty skill file and two default exports in
a declaration. Each guard refuses at plan time what would otherwise fail in
the database mid-apply or be read wrongly.
…er write of the apply takes

A workflow('…') rename was checked only against the names stored now, and a
create only against stored names and other creates. A rename plus a create of
the same name, or two renames onto one name in a folder, both planned clean and
the second write failed part-way (exit 8). The workflow kind now records each
stored workflow's folder and refuses the second write with E_NAME_TAKEN; the
plan-only evaluation reports the name a graph no-op renames to.
…an's findings are cut

capFindings kept the first 1000 errors in order, so a stale, locked or size
error found after 1000 validation errors was cut and the refusal was classed
validation (400, exit 1) instead of 409/423/413. Class-deciding errors now sort
ahead of the rest.
…e applying again continues

A stalled apply whose lease another apply took over stopped with the lease
error mapped to E_APPLY_REFUSED, which the CLI exits 1 (fix the source), though
nothing in the source was wrong and applying again continues. It is now
E_APPLY_FAILED (exit 8).
…d refuses it; a self-calling workflow deletes clean

Covers the edited-parent variant of the delete-in-use check (only the
unchanged parent was tested) and a workflow that runs itself: created in one
apply, and deleted without counting itself as a user.
… drift findings

Groups the files that differ from .sim/written.json by kind and key, reads the
plan findings that mean a resource changed on the server since the base
(I_SERVER_CHANGED/DELETED/MOVED, E_APPLY_STALE, W_UNDECLARED, E_APPLY_NO_BASE),
and classifies the directory as clean, local, drift or conflict. Pure and
client-safe, so the CLI and the server's integration tests share one reading.
status answers, without applying, what an apply would do and whether the server
moved since the get: local changes by resource from file digests against
.sim/written.json (offline), drift and conflicts from a read-only dry run (the
plan's own findings, with every workflow folder sent whole so a workflow edited
in the editor shows), and the one next command. --json/--output json gives a
stable shape; exit 0 clean, 11 local changes only, 12 drift or conflict. A
server that cannot be reached degrades to the local view and says so;
--offline skips it.

diff shows the content: each changed file against its base text (the server's
current text where it still matches the recorded digest, since the CLI keeps
only digests), binary files by size, or with --server every selected file
against the server now. Unified diffs come from jsdiff (already in apps/sim),
colored on a terminal. Paths, folders and kind:key filter.
A directory read by a get, edited on either side and dry-run the way status
sends it, read through the status module: clean, a local edit, a table edited
in the UI, a workflow edited in the editor (only a whole folder reveals it), a
resource added on the server, and conflicts on an edit and on a delete.
Both agent guides run status before apply (diff for the edits, the 11/12 exit
codes, --offline), the workspace guide notes that status, unlike apply, reports
a workflow edited on the server, and the docs pages and SDK README describe the
two commands.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

… does

After the merge, get refuses to write again a file the CLI wrote that is
missing (editedFiles/editedBlobs). status reported nothing for one in a
removed kind folder, so it said clean while get refused: it now reports every
missing recorded file as deleted, blobs included, within --only.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1283 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

Comment on lines +728 to +735
if (detail.publicApi !== undefined) {
await updateWorkflowPublicApi.execute({
principal,
input: { workflowId, isPublicApi: detail.publicApi },
})
}
if (detail.deploy) await deployDraft(change, workflowId, ctx)
if (chatInput) await replaceWorkflowChatDeployment.execute({ principal, input: chatInput })

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Partial deployments block retries

When an existing deployment changes both api.public and its chat, the API flag commits before the chat write. A taken chat identifier, for example, then rejects the chat write but leaves the flag changed. The failed resource returns no new base.

On rerun, the server matches neither the old base nor the desired declaration, so resolveDeclared returns stale—even after the identifier collision is cleared. Commit these settings together, or record enough partial progress to resume without downloading and merging the directory again.

Comment on lines +449 to +453
if (change.detail?.replace) {
// Created first, so a failed create leaves the stored server as it was.
const created = await create()
await remove(current.id)
return written(change.key, created)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Interrupted replacements cannot resume

If the process stops after create() succeeds but before remove(current.id) commits, both MCP servers remain. The source key still belongs to the old server because claimKey runs only after the whole replacement returns.

On rerun, planServer chooses that bound server and plans another replacement. createMcpServerUseCase then rejects the already-created destination URL on every attempt. Recognize and reuse a matching replacement left by an earlier attempt so an interrupted apply can finish.

…plan

E_CREDENTIAL_AMBIGUOUS listed the candidate credentials, and its hint picked
the first, in database listing order, so the finding changed between plans and
the references integration test flaked on it. Candidates now sort by name,
then id.
check (workflows and workspaces) and both source apply commands, dry run or not,
now add diagnostics: [{ code, severity, message, hint?, file?, line?, column?,
path?, kind?, key? }] (the wire FlatDiagnostic without its frame) beside the
counts when the output is json or yaml, so stdout is one document an agent can
parse. Display formats are unchanged and keep the findings on stderr.
…error

A source apply or create the server refuses with an HTTP error (stale, locked,
too large, busy, rejected) printed its findings as text and dropped them from the
error document. In --output json or yaml the document now keeps the refusal's
details, its findings flattened as a success result carries them (without frames):
{ error: { code, message, details: { code, diagnostics } } }. Display formats are
unchanged, and the global error printer is untouched.

This branch was successfully deployed

1 active deployment
Preview — 1514c951 Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant